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Abstract 


This  report  documents  the  work  done  to  process  EC-SWAN  sonobuoy  data  collected 
monthly  over  a  12-month  period  in  late  1995  and  1996.  Nine  or  more  sonobuoys  were 
dropped  at  each  of  four  sites,  and  included  calibrated  Low  Frequency  Analysis  and 
Ranging  (LOFAR),  Vertical  Line  Array  (VLA)  and  Directional  Frequency  Analyzing 
and  Recording  (DIFAR)  types.  The  data  were  processed  to  determine  statistics  for  use 
in  synthetic  sonar  data  modelling.  The  data  were  processed  using  8K  Fast  Fourier 
Transforms  (FFT)  and  ~4  second  incoherent  averages.  The  data  sample  rate  was 
6147.5  Hz.  Statistics  were  then  computed  for  five-minute  blocks,  and  averaged  to 
determine  the  mean  statistic  over  the  one-hour  sample  time.  These  data  are  intended  to 
provide  a  characterization  of  the  probability  density  functions  for  ambient  noise  in 
Canadian  waters  that  can  be  used  for  input  to  sonar  simulation  software. 


Resume 


Le  present  document  fait  etat  des  travaux  effectues  sur  le  traitement  des  donnees  de 
bouees  acoustiques  EC-SWAN  (bruit  ambiant  en  eau  peu  profonde  dans  l'Est 
canadien)  collectees  mensuellement  sur  une  periode  de  12  mois  de  1995  a  1996.  Au 
moins  neuf  bouees  acoustiques  ont  ete  jetees  a  l’eau  a  chacun  des  quatre 
emplacements.  Les  bouees  etaient  des  types  suivants  :  de  telemetrie  et  d’analyse  a 
basses  frequences  (LOFAR),  a  antenne  lineaire  verticale  (VLA)  et  de  detection  et 
d’analyse  directionnelles  a  basses  frequences  (DIFAR).  Les  donnees  ont  ete  traitees 
afin  d'obtenir  les  statistiques  qui  serviront  a  la  modelisation  synthetique  des  donnees 
de  sonar.  Les  donnees  ont  ete  traitees  au  moyen  de  transformes  de  Fourier  rapide 
(FFT)  8K  et  de  moyennes  incoherentes  d'environ  4  secondes.  Le  taux 
d’echantillonnage  des  donnees  etait  de  6  147,5  Hz.  Les  statistiques  ont  ensuite  ete 
calculees  par  blocs  de  cinq  minutes,  et  la  moyenne  statistique  de  Techantillon  a  ete 
determinee  sur  une  heure.  Ces  donnees  serviront  a  obtenir  une  caracterisation  des 
fonctions  de  distribution  de  probability  du  bruit  ambiant  dans  les  eaux  canadiennes, 
qui  peuvent  servir  de  donnees  d'entree  pour  le  logiciel  de  simulation  de  sonar. 
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Executive  summary 


Introduction 

In  past  years,  the  underwater  acoustics  community  has  shifted  focus  from  the  deep 
ocean  to  littoral  water  areas.  Working  with  shallow  water  acoustics  is  extremely 
challenging  since  the  environment  can  change  dramatically  in  both  time  and  space. 

Past  data  collection  exercises,  such  as  the  Eastern  Canada  Shallow-Water  Ambient- 
Noise  (EC-SWAN)  experiment,  were  undertaken  to  study  the  temporal  and  spatial 
variation  of  ambient  noise  in  Eastern  Canadian  waters.  There  is  now  a  renewed  interest 
in  these  statistics,  for  use  in  synthetic  sonar  data  modelling. 

Results 

This  contract  built  a  tool  to  revisit  the  EC-SWAN  dataset  to  provide  a  characterization 
of  the  probability  density  functions  for  ambient  noise  in  Canadian  waters  suitable  for 
input  to  sonar  simulation  software.  This  tool  is  essentially  an  analysis  software  suite 
added  to  the  Software  Tools  for  Analysis  and  Research  (STAR).  This  software 
package  is  extensively  used  at  DRDC  Atlantic  to  efficiently  process  passive  and  active 
sonar  data.  The  EC-SWAN  processing  tool  added  to  the  overall  capability  of  STAR. 

Significance 

This  new  software  package  will  simplify  and  expedite  the  re-processing  of  the 
valuable  EC-SWAN  data,  leading  to  a  set  of  ambient  noise  statistics  that  can  then  be 
implemented  in  any  synthetic  sonar  data  model.  Data  of  this  type  are  presently 
unavailable  in  the  open  literature,  and  are  essential  for  the  study  and  development  of 
sonar  detection  and  tracking  tolls  using  synthetic  data. 

Future  plans 

The  results  of  the  data  processing  will  be  included  in  future  reports. 
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Sommaire 


Introduction 

Au  cours  des  dernieres  annees,  le  milieu  de  l'acoustique  sous-marine  a  deplace  son 
champ  d'interet  de  l’ocean  profond  vers  les  eaux  des  zones  littorales.  L'acoustique  en 
eau  peu  profonde  presente  un  grand  defi,  car  l'environnement  peut  changer  de  facon 
dramatique  dans  le  temps  et  dans  l’espace.  Des  collectes  de  donnees  anterieures,  telles 
que  dans  le  cadre  de  l’experience  EC-SWAN  (bruit  ambiant  en  eau  peu  profonde  dans 
l’Est  canadien),  ont  ete  entreprises  pour  etudier  les  variations  dans  le  temps  et  dans 
l’espace  du  bruit  ambiant  dans  les  eaux  de  l'Est  canadien.  On  s’interesse  a  nouveau  a 
ces  statistiques  pour  les  appliquer  dans  la  modelisation  synthetique  des  donnees  de 
sonar. 

Resultats 

Le  contrat  en  cause  a  donne  lieu  a  la  realisation  d'un  outil  qui  pemiettrait  de  reutiliser 
l’ensemble  de  donnees  EC-SWAN  de  fagon  a  obtenir  une  caracterisation  des  fonctions 
de  distribution  de  probability  du  bruit  ambiant  dans  les  eaux  canadiennes,  qui  peuvent 
servir  de  donnees  d' entree  pour  le  logiciel  de  simulation  de  sonar.  Cet  outil  est 
essentiellement  une  suite  logicielle  ajoutee  aux  outils  logiciels  pour  l’analyse  et  la 
recherche  (STAR).  Ce  progiciel  est  tres  utilise  a  RDDC  -  Atlantique  pour  trader 
efficacement  les  donnees  sonar  passif  et  actif.  L'outil  de  traitement  des  donnees 
EC-SWAN  ameliore  les  capacites  globales  du  STAR. 

Portee 

Ce  nouveau  progiciel  simplifiera  et  accelerera  le  nouveau  traitement  des  precieuses 
donnees  EC-SWAN,  et  qui  menera  a  un  ensemble  de  statistiques  sur  les  bruits 
ambiants  qui  pourra  ensuite  etre  applique  a  tout  modele  synthetique  de  donnees  de 
sonar.  Les  donnees  de  ce  genre  ne  sont  pas  disponibles  dans  la  documentation  non 
classifiee,  et  sont  essentielles  pour  T  etude  et  le  developpement  d'outils  de  detection  et 
de  poursuite  sonar  bases  sur  les  donnees  synthetiques. 

Recherches  futures 

Les  resultats  du  traitement  des  donnees  figureront  dans  les  rapports  a  venir 
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1.  Introduction 


This  final  report  outlines  the  work  done  under  Noise  Monitoring  Software  Regional  Individual 
Standing  Offer  (RISO)  W7707-032293/001/HAL,  Call-up  Requisition  No.  W7707-04-2681.  This 
work  was  performed  for  Defence  R&D  Canada  (DRDC)  -  Atlantic  under  the  direction  of  the 
Scientific  Authority  (SA),  Francine  Desham ais,  from  approximately  December  2004  to 
March  2005. 


1.1  Overview 

The  objective  of  this  call-up  was  to  simplify  the  calculation  and  presentation  of  ambient  noise 
statistics  that  could  be  used  for  input  in  sonar  simulation  software.  This  was  done  by  using  generic 
software  components  available  in  the  Signal  Processing  Packages  (SPPACS)  to  batch  process  raw 
data,  outputting  power  spectral  data  incoherently  averaged  over  four  seconds.  The  processed  data 
were  also  calibrated  for  processing  and  sonobuoy  gain  and  stored  in  DREA  .pwr  formatted  files.  A 
second  processing  stream  was  used  to  compute  third-octave  averages  starting  at  6  Hz  for  the  same 
data.  The  processed  data  were  then  read  into  Interactive  Data  Language  (IDL)  in  five-minute 
chunks,  and  the  mean,  median,  skew,  kurtosis  and  standard  deviation  computed  for  selected 
frequencies  and  the  third-octave  averages.  All  of  these  statistics  were  computed  on  Log  scaled 
data.  These  5-min  chunks  were  then  averaged  over  12  samples,  or  as  many  as  were  available  in 
each  data  file,  to  produce  an  average  statistic.  There  were  usually  ten  sonobuoys  per  site  with  four 
sites  being  sampled  per  flight.  The  flights  occurred  over  the  course  of  a  year  from  December  1995 
to  December  1996. 

This  report  is  broken  into  five  main  sections.  The  remainder  of  this  section  provides  background 
on  the  software  used  during  the  call-up.  Section  2  provides  an  overview  of  the  call-up's  software 
development  requirements  and  work  performed  to  meet  those  requirements.  Section  3  provides  an 
overview  of  data  content.  All  raw  and  processed  data  were  copied  to  a  hard  drive  accessible  to  the 
SA.  The  raw  data  alone  consumed  more  than  30  GB  of  storage.  Section  4  details  software 
configuration  management  processes  and  the  versions  used  for  this  call-up.  Section  5  provides  a 
summary  of  current  software  issues.  None  of  the  current  software  issues  affect  this  call-up. 

1.2  Background 

The  data  processing  and  analysis  work  was  performed  using  two  software  suites:  Software  Tools 
for  Analysis  and  Research  (STAR)  and  SPPACS.  An  overview  of  the  two  software  suites  is 
provided  in  the  following  subsections. 

The  STAR  and  SPPACS  suites  are  configuration  controlled  using  the  concurrent  versioning 
system  (CVS),  and  issue  and  enhancement  idea  tracking  is  affected  using  the  Bugzilla  issue 
tracking  software.  CVS  is  a  repository  that  allows  developers  to  check-in  revisions  to  software  and 
documentation  where  they  are  archived  in  a  common  database.  The  tool  allows  all  previous 
versions  of  the  software  to  be  maintained  and  aids  resolution  of  new  issues,  while  ensuring  that 
current  builds  of  the  software  are  readily  accessible  to  users  and  developers  alike.  Bugzilla  is  a 
web  accessible  database  that  offers  both  user  and  developer  input  to  issues,  priorities  and 
solutions.  It  provides  coherent  tracking  and  recording  of  an  issue  over  its  entire  lifecycle. 


DRDC  CR  2005-126 


1 


STAR  and  SPPACS  components  are  documented  in  a  combination  of  formats,  each  with  their  own 
puipose.  Microsoft  Word  documents  are  maintained,  which  describe  functionality  and  algorithms 
of  components.  These  are  primarily  intended  for  the  end  user.  Enterprise  Architect  (EA)  files  are 
maintained,  and  document  software  design,  interaction  and  dependencies.  EA  design  information 
is  intended  primarily  for  developers.  Hypertext  Markup  Language  (HTML)  library  documentation 
is  being  developed,  which  provides  automatic  extraction  of  the  routine’s  Application  Program 
Interface  (API),  puipose  and  description.  This  documentation  is  maintained  to  assist  developers  in 
familiarizing  themselves  with  the  existing  libraries  and  components,  and  is  intended  to  support 
and  encourage  software  reuse.  Some  users  may  also  wish  to  refer  to  this  information  for  use  in 
their  own  custom  applications.  SPPACS  also  provides  HTML  and  man  page  user  documentation 
for  each  module. 

The  most  current  status  of  the  SPPACS  and  STAR  suites  can  be  found  at  https://stai~.iotek.ns.ca. 
Users  are  also  encouraged  to  refer  to  the  electronic  documentation  provided  with  the  software 
distribution  for  up-to-date  information. 

1.2.1  STAR 

The  STAR  suite  was  developed  to  support  general  research  and  analysis  objectives  at  DRDC 
Atlantic.  The  primary  objectives  of  the  STAR  suite  are: 

•  Provide  scientific  grade  analysis  tools  that  allow  for  efficient,  detailed  quantitative  and 
qualitative  analysis  of  a  data  set. 

•  Support  synergy  between  DRDC  groups  and  the  Department  of  National  Defence  (DND)  by 
providing  a  common  software  base  for  analysis.  This  synergy  encourages  inter-group 
communication  and  simplifies  user  training,  analysis  process  development,  documentation 
and  data  portability. 

•  Support  cost  and  analysis  efficiency  by  providing  software  reuse  and  common  tools  and  data 
formats.  Examples  of  efficiency  would  be  using  the  output  of  analysis  from  one  group  to  feed 
the  inputs  of  another,  or  using  common  software  components  to  lower  development  cost  of 
several  custom  analysis  tools. 

All  STAR  components  are  currently  implemented  using  IDL,  though  the  design  is  not  restricted  to 
IDL.  The  name  STAR  reflects  the  generic  nature  of  the  software.  Applications  in  the  STAR  suite 
are  built  using  a  combination  of  reusable  and  custom  components  that  meet  the  requirements  of 
each  application.  The  layered  design  and  common  components  allow  for  rapid  and  logical 
development  of  new  capabilities.  Though  currently  focused  on  sonar  data  processing  and  analysis, 
the  tools  are  capable  of  expanding  to  meet  other  analysis  and  research  requirements. 

1.2.2  SPPACS 

SPPACS  is  a  group  of  software  programs  that  are  based  on  the  C  programming  language  and  are 
implemented  on  Linux-based  personal  computers  (PC).  Each  program  provides  a  specific 
processing  function  and  a  series  of  programs  can  be  chained  together  to  create  a  custom- 
processing  stream  using  the  command  line  or  scripts.  The  output  from  SPPACS  is  stored  in  DRDC 
Atlantic  formatted  data  files.  SPPACS  has  slowly  evolved  to  its  present  day  state  due  to  the  efforts 
of  several  MacDonald  Dettwiler  and  Associates  Ltd.  (MDA)  personnel  over  the  last  4  years. 
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SPPACS  has  been  used  to  perform  a  number  of  mid-trial  and  post-trial  processing  functions,  such 
as  the  post-trial  study  of  multistatic  trial  data  and  the  mid-trial  analysis  of  the  Q265  sonobuoy  test 
trial.  SPPACS  only  performs  data  manipulation  and  does  not  provide  an  interface  to  examine  the 
results.  The  processed  data  output  is  often  imported  into  other  applications  that  enable  data  display 
and  are  used  to  perform  the  detailed  analysis  of  the  results.  One  example  of  such  an  application  is 
the  STAR  suite. 

The  SPPACS  software  suite  consists  of  two  types  of  software.  One  type  is  runtime  executables 
that  can  be  used  to  process  DRDC  Atlantic  data  files  in  a  number  of  ways,  including  data 
management  and  signal  processing.  Each  program  performs  a  specific  function  and  the  programs 
are  designed  so  that  they  can  be  used  in  conjunction  to  perform  more  complex  processing  tasks. 
The  software  has  proven  to  be  very  useful  in  simplifying  data  management  and  sonar  processing 
tasks  by  providing  a  set  of  tools  from  which  to  build  the  necessary  processing  streams.  These 
streams  can  be  run  from  the  command  line  or  assembled  into  scripts  to  perform  batch-processing 
tasks  allowing  for  large  amounts  of  data  to  be  automatically  processed.  The  second  form  of  the 
software  is  a  group  of  library  functions  that  can  be  used  by  other  programs  to  efficiently  perform 
standard  tasks.  These  library  functions  are  extensively  used  by  the  runtime  software,  but  can  also 
be  used  for  other  applications.  There  are  now  three  types  of  libraries.  The  first  are  utility  routines 
for  performing  tasks,  such  as  header  manipulation  and  command  line  parsing.  The  second  are 
signal  processing  modules  termed  Signal  Processing  Library  SPLIB.  These  are  low-level  modules, 
each  performing  a  low  level  signal -processing  task.  A  new  SPPACS  module  typically  consists  of 
one  or  more  SPLIB  modules  linked  together  with  an  SPPACS  user  interface.  The  final  library  type 
is  a  sonar  processing  module  termed  sonar  library  (SONLIB).  These  are  more  complex  modules 
that  combine  several  SPLIB  modules  to  create  a  complex  sonar  module,  such  as  passive 
processing.  Separating  the  SPLIB  and  SONLIB  modules  from  SPPACS  generated  more 
generically  reusable  software.  SPLIB  and  SONLIB  are  independent  of  the  data  header  format, 
timestamping  method,  etc.,  and  are  suitable  for  integration  in  real-time  processing  systems. 

SPPACS  is  also  supported  by  a  set  of  signal  processing  libraries  known  as  the  Lastest  Fourier 
Transform  in  the  West  (FFTW).  These  free,  open-source  libraries  provide  optimized  signal 
processing  functions  helping  to  ensure  that  the  SPPACS  software  runs  as  efficiently  as  possible, 
while  providing  a  significant  reduction  in  coding  effort. 


DRDC  CR  2005-126 


3 


2.  Work  Overview 


This  section  presents  an  overview  of  the  tasked  work  and  a  summary  of  the  work  that  was 
completed.  The  development  requirements  for  this  call-up  are  listed  in  Table  1.  All  requirements 
were  successfully  completed. 


Table  1.  Requirements 


TASK 


DESCRIPTION 


Task  1 :  Support  Data  Processing 


The  contractor  shall  support  processing  of  provided 
trial  data  to  produce  LOFARGRAM  of  the  data.  This 
processing  will  be  tuned  to  provide  statistical  analyses 
of  the  data  using  algorithms  provided  by  the  SA. 


Task  2:  Requirements  Analysis  and 
Design 


Task  3:  Implement  and  Processing 
Requirements 


The  contractor  shall  meet  with  the  SA  prior  to  the 
implementation  to  identify  which  algorithms  will  be 
implemented  and  the  level  of  user  interaction  required. 
User  interaction  will  include  setting  of  algorithm 
parameters,  modification  of  which  algorithms  are  used 
and  modification  of  which  data  are  passed  to  the 
algorithms.  (NOTE:  The  statistical  algorithms  must 
also  function  on  a  ‘headless’  system  for  batch  runs.) 

The  contractor  shall  address  the  requirements,  detailed 
in  the  following  list,  and  as  prioritized  by  the  SA,  by 
enhancing  the  STAR  Suite. 

Data  survey: 

-  Download  data  from  DLT  tapes. 

-  Process  power  files  with  settings  defined  by 

the  SA. 

-  From  LOFARGRAMs,  identify  periods  with 

suitable  ambient  noise  data  [-1.25  hr 
periods].  Annotate  period  for  presence/ 
absence  of  ship  noise  or  other  events. 

Statistical  algorithms : 

-  Calculate  mean  and  median  noise  levels, 

standard  deviation,  skew  and  kurtosis  for  a 
user-selected  intervals  [5  min],  and  a  user- 
determined  set  of  frequencies  and 
band  widths. 


-  Calculate  probability  distribution  surfaces 
(probability  vs  noise  level  and  frequency) 
for  the  same  intervals.  PDFs  should  be 


normalized  so  that: 


Jf(y)dy  =  h 
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Table  1.  Requirements 


TASK 

DESCRIPTION 

-  Calculate  the  10,  50  and  90%  percentile  of 
ambient  noise  spectra  for  the  same  intervals. 

-  Tabulate  above  results. 

-  Plot  above  results  on  1  page  for  a  given 
sonobuoy. 

-  For  the  entire  (-1.25  hr),  calculate  the  overall 
mean,  median,  standard  deviation,  average 
skewness  and  kurtosis  and  decorrelation 
time. 

-  Tabulate  the  overall  results. 

The  SA  will  indicate  where  algorithm  and  design 
review  is  required  and  the  SA  shall  approve  or  provide 
those  algorithms  and  designs  prior  to  implementation. 
Algorithms  should  be  designed  to  allow  for  an  arbitrary 
choice  of  processing  parameters,  such  as  binwidth, 
period  selection  for  various  statistics,  etc. 

Data  Processing: 

The  contractor  shall  process  a  subset  of  the  EC-SWAN 
dataset,  to  be  provided  by  the  SA.  The  complete  EC- 
SWAN  dataset  includes  12  sets  of  1  to  1.5  hour  over 

4  sites  (9  sonobuoys/  site). 

Task  4:  Perform  Testing 

The  contractor  shall  develop  test  plans  and  test  the 
software  to  the  satisfaction  of  the  SA  and  where 
possible,  automate  those  tests  to  allow  for  rapid 
regression  and  installation  testing.  The  SA  shall  have 
the  option  to  review  tests  prior  to  testing. 

Task  5:  Software  Maintenance  and 
Design  Improvement 

As  required,  the  contractor  will  troubleshoot  and  repair 
defects  in  the  software. 

As  time  permits  and  as  authorized  by  the  SA,  the 
contractor  will  examine  the  evolving  software  design 
and  implement  improved  design  features  helping  to 
ensure  the  long-term  viability  and  reusability  of  the 
software. 

As  required  and  as  specified  by  the  SA,  the  contractor 
will  add  enhanced  functionality  to  the  software  within 
the  limits  of  the  call-up  funding. 

Task  6:  Maintain  Configuration 
Management 

The  contractor  shall  implement  configuration 
management  of  the  IDL  software  using  the  Concurrent 
Versioning  System  (CVS)  and  provide  software 
deliverables  by  checking  them  into  CV S  and  then 
demonstrating  a  successful  checkout  and  run. 
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Table  1.  Requirements 


TASK 

DESCRIPTION 

Task  7:  Track  Software  Issues 

The  contractor  shall  use  software  issue  tracking  and 
resolution  management  using  the  Bugzilla  issue¬ 
tracking  tool. 

Web-based  entry  and  review  of  issues  from  DRDC 
Atlantic  shall  be  made  available  to  the  SA  on  the 

STAR  Portal  web  site. 

Task  8:  Generate  Documentation 

The  following  details  shall  be  documented  in  the 
contractor  report: 

a.  As  implemented  algorithms  and  design 

b.  Testing  performed 

c.  User  interface  changes 

2.1  Data  Processing 

Data  processing  requirements  were  as  follows: 

•  Download  data  from  Digital  Linear  Tapes  (DLT) 

•  Process  power  files  with  settings  defined  by  the  SA 

•  From  LOFARGRAMs,  identify  periods  with  suitable  ambient  noise  data  [-1.25  hour  periods]. 
Annotate  period  for  presence/absence  of  ship  noise  or  other  events. 

A  number  of  small  changes  were  made  to  this  task  during  the  call-up. 

•  It  was  found  that  the  data  were  on  Exabyte  tapes.  A  tape  drive  was  taken  on  loan  from  DRDC 
and  the  tapes  were  transcribed  to  a  removable  hard  drive  with  a  Universal  Serial  Bus  (USB) 
interface.  This  allowed  all  processed  and  raw  data  to  be  quickly  transferred  back  onto  DRDC 
servers  after  call-up  completion. 

•  It  was  decided  to  process  the  data  using  8K  FFTs,  50%  overlap,  Hanning  windows  and  five 
incoherent  averages.  Third -octave  processing  was  to  start  at  6  Hz.  The  SA  provided  calibration 
files  for  all  buoy  types.  Two  scripts  were  created  to  perform  power  spectra  (process_spectra)  and 
third-octave  processing  (process_third_octave),  but  they  later  had  to  be  modified  to  account  for 
changes  in  buoy  type  to  recording  channel  assignments  over  the  flights.  This  was  handled  by 
passing  in  parameters  describing  a  list  of  the  channels  associated  to  each  buoy  type,  and  using  the 
update_proc  script  as  a  master  script.  All  processing  scripts  can  be  found  in  the  scripts  directory 
of  the  trial.  All  calibration  files  can  be  found  in  the  cal_files  directory  of  the  trial. 

•  During  the  call-up  it  was  found  that  a  data  survey  had  already  been  conducted  and  ‘deleted 
spectra’  were  interpreted  to  be  times  with  bad  data.  An  option  was  added  to  the  IDL  processing  to 
ignore  those  spectra  in  hourly  average  computation.  Regardless,  LOFARGRAMs  can  be  viewed 
in  STAR  by  the  SA  from  the  processed  .pwr  files  produced. 
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2.2  Statistical  Processing  and  Output  Formatting 


Statistics  processing  requirements  were  as  follows: 

•  Calculate  mean  and  median  noise  levels,  standard  deviation,  skew  and  kurtosis  for  a  user- 
selected  interval  [5  minutes],  and  a  user-determined  set  of  frequencies  and  bandwidths. 

•  Calculate  probability  distribution  surfaces  (probability  versus  noise  level  and  frequency)  for 

oo 

the  same  intervals.  PDFs  should  be  normalized  so  that:  J*  / ( y)dy  =  1. 


•  Calculate  the  10,  50  and  90  percentiles  of  ambient  noise  spectra  for  the  same  intervals. 

•  Tabulate  the  results. 

•  Plot  results  on  a  single  page  for  a  given  sonobuoy. 

•  For  the  entire  (-1.25  hour),  calculate  the  overall  mean,  median,  standard  deviation,  average 
skewness  and  kurtosis,  and  decorrelation  time. 

•  Tabulate  the  overall  results. 

User  frequencies  were  selected  to  be  20,  63,  500,  800,  1260  and  2000  Hz.  The  bins  closest  to  these 
frequencies  were  used.  It  was  decided  that  the  .pdf  information  would  only  be  provided  in  the 
plots,  but  that  they  would  also  be  available  from  an  IDL  structure  at  run-time  for  manual  analysis. 

Using  the  STAR  library  of  applications,  a  more  generic  statistics  analysis  window  application  was 
created  to  display  the  results  of  the  ambient  noise  statistics  calculations.  This  application  was 
written  to  allow  the  user  to  modify  their  selections  at  run  time  without  further  software 
development.  In  the  start-up  script,  the  user  may  define: 

•  An  arbitrary  list  of  frequencies  or  request  all  frequencies. 

•  An  arbitrary  list  of  receivers  (data  channels)  or  all  receivers. 

•  There  is  a  flag  to  define  whether  the  frequencies  should  be  calculated  using  a  third  octave 
formula,  as  there  is  no  distinct  definition  of  third-octave  data  in  a  DREA  header. 

•  The  time  interval  in  seconds  over  which  to  process  a  data  chunk,  in  this  case  300. 

•  The  display  spacing  for  data  sets.  This  is  an  integer  to  specify  which  chunks  to  use.  For 
example,  if  set  to  5,  every  fifth  processed  chunk  will  be  displayed  on  the  summary  plots.  If  set 
to  1,  every  chunk  will  be  plotted,  though  the  resulting  plots  are  likely  to  be  very  compressed. 

•  The  overall  time  range  to  calculate  statistics  may  be  set  in  the  start-up  script.  This  can  be  used 
to  select  only  a  portion  of  the  data. 
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•  The  PDF  starting  level,  spacing  and  ending  level  may  be  set. 

•  The  user  may  also  use  a  set  of  flags  to  disable  the  text  file  output  (tabular  summaries),  the 
postscript  summary  plot  output,  or  the  display  window. 

On  execution  of  the  script,  all  statistics  for  all  data  sets  of  selected  frequencies  and  receivers  will 
be  calculated  and  stored  in  a  memory  structure.  The  output  text  tables,  including  overall  summary 
tables  and  postscript  summaiy  plots,  are  then  generated  automatically.  Finally,  the  summary  plots 
may  then  be  reviewed  individually  in  the  statistics  analysis  window.  Selecting  the  ‘View  Next’ 
button  displays  the  summaiy  for  the  next  receiver.  The  user  also  has  the  option  to  selectively  save 
a  plot  to  a  custom  Postscript  file  and  suspend  the  in-memory  structure  to  IDL.  This  suspend 
function  allows  the  user  to  access  the  in-memory  data  from  the  IDL  command  line,  or  from  a 
custom  script. 

Figure  1  shows  a  sample  display  window.  The  following  statistics  are  plotted  for  the  SA  selected 
frequencies  for  every  third  chunk: 

•  Mean 

•  Standard  Deviation 

•  Skew 

•  Kurtosis 

•  PDF 

•  10,  50,  90  percentiles  from  the  PDF  (Solid,  Dotted  and  Dashed  lines,  respectively) 

The  filename,  processor  channel  and  time  are  down  in  the  first  plot’s  title.  The  remaining  plot 
titles  show  the  time  range  in  minutes  from  the  start  of  the  file  that  was  used  to  generate  the  plot. 
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Figure  1.  Statistics  Analysis  Display 
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The  postscript  output  files  are  identical  to  the  screenshot,  except  the  PDF  is  greyscale  and  the 
background  is  white  with  black  text  and  graphics.  A  sample  of  the  output  tables  is  shown  in 
Figure  2.  They  are  delimited  American  Standard  Code  for  Information  Interchange  (ASCII)-based 
text  files  that  can  be  read  into  applications,  such  as  Microsoft  Excel,  for  custom  analysis. 


#File  Created : /opt /ssp/ data/ ambient_noise /processed/ trial lomni- 
527b/spectra_data/ jul04_lb . pwr_noise_stats . txt ,  on:Wed  Mar  30  23:43:32  2005 

♦Source  File : /opt/ssp/data/ambient_noise/processed/triallomni- 
527b/spectra_data/ jul04_lb . pwr.  Data  start  time:  July  4  15:19:00  1996 

♦Recorder  Receiver  RF  Latitude 
♦triallomni-527b  1  OM_N_B 

♦triallomni-527b  2  OM_E_l 

♦triallomni-527b  3  OM_W_l 

♦Frequency (tonal) :  20.261671,  Beam:  0.0000000 

♦RF: Receiver  Sample  Mean  Median  StdDev  Skew  Kurtosis 

OM_N_B : 1  0:5  85.509453  85.462433  3.1743660  -0.82637519  3.0205503 

OM_N_B : 1  5:10  84.708450  84.999207  2.3486032  -0.35854453  -0.42521572 

OM_N_B : 1  10:15  85.531601  85.689682  2.0663860  -0.36596778  0.60989571 

OM_N_B : 1  15:20  85.187042  85.167458  2.7759728  -0.15457967  0.96901655 

OM_N_B : 1  20:25  85.988091  86.290359  2.5977252  -0.47521719  0.45095468 

OM_N_B : 1  25:30  85.296471  85.115044  4.3040609  -0.67177862  3.2522545 

OM_N_B : 1  30:35  69.115372  79.069870  17.151575  0.023129262  -1.9521452 

OM_N_B : 1  35:40  51.796375  51.903130  0.75303471  -0.15878107  -0.62454915 

OM_N_B : 1  40:45  51.435127  51.448769  0.67389983  -0.22400841  -0.37052369 

OM_N_B : 1  45:50  51.585316  51.603889  0.79031360  1.4706811  8.6684132 

OM_N_B : 1  50:55  51.965252  51.834976  1.6483419  7.8789630  67.763794 

♦RF:Receiver  Sample  Mean  Median  StdDev  Skew  Kurtosis 

OM_E_l  : 2  0  : 5  61.893429  62.447556  2.3298728  -0.49095577  0.41027117 

OM_E_l  : 2  5  :10  61.682838  61.855888  2.1217639  -0.45029375  0.061904907 

OM_E_l  : 2  10  : 15  61.743435  62.171509  2.5413525  -0.59235471  0.80893350 

OM_E_l  : 2  15  :20  62.040924  61.925331  2.8633842  0.54202247  2.7351904 

OM_E_l  : 2  20  :25  61.718792  61.665970  2.3424962  -0.028601099  -0.0049319267 

OM_E_l  : 2  25  :30  62.946751  62.900177  2.2354417  0.51447237  0.14676499 

OM_E_l  : 2  30  :35  62.022194  62.435600  1.8616357  -0.52803302  -0.64442730 

OM_E_l  : 2  35  :40  62.004192  62.003716  2.4315717  1.3564923  4.3333879 

OM_E_l : 2  40:45  62.063938  62.227768  2.3041084  0.66160828  2.2344894 

OM_E_l : 2  45:50  64.687500  64.422134  3.1511843  0.76120478  2.0443554 

OM_E_l : 2  50:55  63.698215  63.250832  4.3913574  2.0888584  7.4132318 

♦RF: Receiver  Sample  Mean  Median  StdDev  Skew  Kurtosis 

OM_W_l:3  0:5  62.544231  62.447762  2.8860269  0.094522350  0.39098573 

OM_W_l:3  5:10  62.055416  62.092567  2.4406767  0.26085782  1.0372601 

OM_W_l:3  10:15  61.969933  62.107811  2.0345299  -0.24050899  0.40238070 

OM_W_l:3  15:20  61.887089  61.812531  2.2967062  0.36986551  0.83945417 

OM_W_l:3  20:25  61.919964  62.133865  2.2541733  -0.081670716  -0.16462708 

OM_W_l:3  25:30  61.987724  61.873497  2.4649143  0.23451820  -0.039600849 

OM_W_l:3  30:35  61.118908  61.677765  2.3606203  -0.39654845  -0.26389527 

OM_W_l:3  35:40  61.949940  61.950729  2.3073370  0.46918571  1.4626527 

OM_W_l:3  40:45  61.972298  61.957443  2.6926897  1.3791738  5.1065559 

OM_W_l:3  45:50  64.202072  62.588913  5.7590089  1.8694299  3.1886806 

OM_W_l:3  50:55  63.774746  62.688965  5.0381756  2.4106739  5.9048653 

♦Hourly  averaged  summary  data  excluding  data  sets:  12 

♦RF:Receiver  Mean  Median  StdDev  Skew  Kurtosis  Decorrelation_Time (s) 

0M_N_B : 1  71.647141  72.598620  3.4803890  0.55795644  7.3965859  12.568514 

OM_E_l : 2  62.409292  62.482407  2.5976517  0.34858365  1.7762882  19.231341 

OM_W_l:3  62.307484  62.121077  2.9577144  0.57904536  1.6240647  14.688505 

Figure  2.  Postscript  Output  Sample 


Longitude 

45.500000  -50.000000 
45.500000  -50.000000 
45.500000  -50.000000 
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Each  tabular  output  follows  the  same  format.  The  first  line  of  the  file  indicates  the  name  of  the 
original  output  file  and  the  date  that  the  data  were  processed  by  IDL.  The  second  line  indicates  the 
processed  data  file  used  for  source  data,  and  the  date  and  time  from  the  source  file  header.  The 
next  group  indicates  the  receivers  and  their  position  as  entered  into  the  non-acoustic  data  (NAD) 
for  the  trial.  In  this  case,  it  is  the  nominal  center  of  the  area.  The  radio  frequency  (RF)  number 
actually  shows  the  receiver  name,  as  provided  by  the  EC-SWAN  team.  After  this,  tabular  statistics 
for  each  frequency  are  provided.  In  this  case  it  is  tonal  data  indicated  by  the  text  on  the  first  line. 
The  beam  direction  is  also  specified,  in  case  beamformed  statistics  are  desired  at  a  later  date.  Each 
receiver’s  statistics  are  tabulated,  followed  by  a  summary  for  all  receivers  at  the  end  of  the  block. 
The  remainder  of  the  file,  not  shown,  provides  tables  for  the  remaining  frequencies.  An  identical 
output  is  produced  for  third  octave  data,  except  that  the  displayed  frequency  is  the  center  of  the 
third  octave  band,  computed  as  the  sixth  octave,  and  the  annotation  marks  it  as  such  “(1/3-oct 
band)”. 

2.3  STAR  Scripts 

STAR  applications  are  generally  stalled  using  a  custom  script.  These  scripts  are  built  from  a 
template  provided  with  the  software  distribution,  and  allow  users  to  define  a  variety  of  custom 
processing  and  viewing  options,  such  as  those  shown  previously.  Other  generic  options,  such  as 
font  size,  can  also  be  set. 

Several  STAR  scripts  were  provided  along  with  the  trial  data.  A  short  description  of  each  follows: 

•  Stats_anal.pro:  This  script  could  be  used  to  provide  an  overview  of  all  data.  It  processes 
statistics  for  all  frequencies  in  its  current  state,  so  it  does  take  some  time  to  run. 

•  Tonal_anal.pro:  This  script  is  used  to  produce  output  for  each  selected  tonal  frequency.  In  its 
current  state,  it  provides  a  graphical  user  interface  (GUI),  postscript  output  and  tabular  output. 

•  Oct_anal.pro:  This  script  is  used  to  produce  output  for  all  computed  third-octave  bands.  In  its 
current  state,  it  provides  a  GUI,  postscript  output  and  tabular  output. 

•  Gram_view.pro:  This  script  launches  the  STAR  power_spectral  application,  which  can  be 
used  to  view  LOFARGRAM  data.  It  is  useful  as  it  provides  a  common  view  of  the  data  that  can 
be  used  to  understand  anomalies,  such  as  those  that  may  be  caused  by  close  range  contacts  or 
aircraft  over-flights. 

•  Ambient_anal.pro:  This  script  launches  a  STAR  application  designed  to  view  .pwr  formatted 
data  files  from  a  number  of  perspectives.  For  example,  you  can  plot  many  time  records  of  a  single 
bin  or  a  single  time  record  for  all  bins.  You  can  also  plot  a  single  time -bin  point  over  all 
receivers.  This  could  be  useful  for  more  qualitative  analysis  of  the  data. 

•  File_dur_an.pro:  This  is  a  simple  script  that  was  used  to  determine  the  file  length  for  all 
transcribed  files.  Its  output  is  shown  in  section  3. 
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2.4  Software  Reuse 


The  LOFARGRAM  viewing  and  ambient  noise  analysis  tools  were  not  generated  as  part  of  this 
call-up  but  were  available  for  use  because  of  the  common  approach  taken  in  executing  this  call-up. 
This  is  a  direct  example  of  the  utility  of  selecting  a  common  set  of  analysis  tools  and  building  on 
them  for  custom  analysis  requirements. 

Other  capabilities  used  during  this  call-up  also  came  from  reusable  components  generated  during 
other  call-ups.  Without  these,  it  would  have  been  impossible  to  provide  an  application  with  a 
structure  as  seen  here.  Instead  a  “one-of”  script  would  have  been  required  that  only  met  the 
contract  requirements  and  did  not  support  further  re-use.  Some  of  the  reusable  components  used 
include: 

•  DRDC_file_io  /  Power_data:  Several  libraries  of  functions  designed  to  read  segments  of  data 
from  DREA  formatted  data  files,  handle  header  formats,  etc. 

•  Time:  A  simple  library  to  handle  manipulation  and  formatting  of  time  based  data.  It  also 
includes  utility  functions  to  convert  to  and  from  text,  and  DREA  header  formats. 

•  Tactical_database:  This  is  the  component  that  was  used  to  parse,  store  and  provide  query 
access  to  receiver  names.  It  allowed  the  application  to  run  genetically  and  simply  fill  in  this  data, 
as  required. 

•  Position_model:  This  application  caches  currently  displayed  or  otherwise  required  receivers 
for  quick  access. 

These  modules  also  have  other  capabilities  that  were  not  used  during  this  call-up,  and  still  more 
modules  are  developed  and  tested  so  that  they  can  be  used  in  other  work. 


2.5  Further  Information 

A  detailed  description  of  the  technical  aspects  related  to  the  work  performed  under  this  contract 
can  be  found  in  the  primary  STAR  reference,  “The  Scientific  Tools  for  Analysis  Research  -  Data 
Analysis  and  Technical  Manual”.  This  document  provides  details,  such  as  the  algorithms  used  to 
perform  the  various  analysis  measurements,  and  documents  the  analysis  process  used  to  process 
and  analyze  the  data  for  this  call-up.  The  same  process  can  be  used  for  all  work  with  STAR  and  it 
can  offer  significant  improvements  in  efficiency,  if  followed.  A  soft  copy  of  the  most  recent 
version  of  the  document  can  be  found  in  draft  form  in  the  distribution  directory  under 
acoustics/doc/an  alysis_tools/STAR_an  alysis_technical_manual.doc. 
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3.  Data  Content 


This  section  summarizes  content  of  the  trial  data.  Detailed  processing  results  have  been  provided 
electronically,  as  the  dataset  is  much  too  large  to  efficiently  format  and  publish  in  a  report.  The  SA 
and  project  manager  agreed  that  attempting  to  reformat  results  into  a  report  would  not  be  a 
productive  use  of  contract  funds.  Detailed  information  on  which  sonobuoy  types  were  recorded 
and  where  they  were  located  is  available  from  the  SA. 

Table  2  summarizes  the  data  transcribed  from  Exabyte  tape.  Note  that  one  event  was  copied  from 
CD  and  several  tapes  had  read  errors.  The  duration  of  the  successfully  transcribed  data  is  also 
provided. 


Table  2.  Data  Summary 


FILE  NAME: 

FILE  DURATION  (MINUTES): 

NOTES: 

f1st_dec18.dat 

61.311886 

OK 

f1s2_dec18.dat 

60.551907 

Bad  Date  and  Time  -  patched  with 

18  Dec  95  12:00:00 

f1s3_dec18.dat 

60.791353 

OK 

f1s4_dec18.dat 

60.531086 

OK 

f2s1Jan18.dat 

60.093838 

OK 

f2s2Jan18.dat 

59.979321 

OK 

f2s3Jan18.dat 

60.378396 

OK 

f2s4Jan18.dat 

60.857287 

OK 

f3s1_feb21  .dat 

59.802339 

OK 

f3s2_Feb21  .dat 

50.033664 

I/O  Error  after  564M 

f3s3_Feb21  .dat 

61.391701 

OK 

f3s4_Feb21  .dat 

59.948089 

OK 

f4s1_Mar14.dat 

59.909916 

OK 

f4s2_Mar14.dat 

59.875214 

OK 

f4s3_Mar14.dat 

48.909312 

OK 

f4s4_Mar14.dat 

60.492913 

OK 

f5s1_Apr30.dat 

60.669895 

OK 

f5s2_Apr30.dat 

60.978745 

OK 

f5s3_Apr30.dat 

60.208355 

OK 

f5s4_Apr30.dat 

60.878108 

OK 

f6s1_Jun05.dat 

60.923221 

OK 

f6s2_Jun05.dat 

60.888519 

OK 

f6s3_Jun05.dat 

60.860757 

OK 

f6s4_Jun05.dat 

60.211825 

OK 
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Table  2.  Data  Summary 


FILE  NAME: 

FILE  DURATION  (MINUTES): 

NOTES: 

f7s1_Jul04b.dat 

58.000036 

OK  -  from  CD 

f7s2_Jul04b.dat 

40.091477 

OK  -  from  CD 

f7s3_Jul04b.dat 

58.000036 

OK  -  from  CD 

f7s4_Jul04b.dat 

58.000036 

OK  -  from  CD 

f8s1_Jul29.dat 

60.149361 

OK 

f8s2_Jul29.dat 

60.367985 

OK 

f8s3_Jul29.dat 

60.191004 

OK 

f8s4_Jul29.dat 

60.104249 

OK 

f9s1_Aug21  .dat 

60.066076 

OK 

f9s2_Aug21  .dat 

60.111189 

OK 

f9s3_Aug21  .dat 

61.971228 

OK 

f9s4_Aug21  .dat 

60.225706 

OK 

f10s1_0ct30.dat 

48.940544 

I/O  Error  -  551 M 

f10s2_0ct30.dat 

50.241877 

I/O  Error -566M 

f10s3_0ct30.dat 

59.382442 

OK 

f10s4_0ct30.dat 

60.111189 

OK 

f11s1_Nov04.dat 

60.399217 

OK 

f11s2_Nov04.dat 

60.329813 

OK 

f11s3_Nov04.dat 

60.28817 

OK 

f11s4_Nov04.dat 

70.296291 

OK 

f12s1_Dec10.dat 

60.2847 

OK 

f12s2_Dec10.dat 

60.555377 

OK 

f12s3_Dec10.dat 

60.173653 

OK 

f12s4_Dec10.dat 

49.985081 

OK 
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4.  Maintain  Configuration  Management 


STAR  and  SPPACS  are  maintained  using  CVS.  The  most  recent  release  version  is  maintained  and 
bug  fixes  are  applied  to  that  version,  as  required,  ensuring  that  a  stable  release  is  always  available. 
Simultaneously,  software  enhancements  are  applied  to  the  development  version  and  bug  fixes  are 
merged  with  this  version.  Once  a  call-up  nears  completion,  or  a  release  of  the  software  is 
otherwise  required,  a  new  release  version  is  branched  off  of  the  development  stream  for  final 
integration,  release  testing  and  delivery. 

STAR  (includes  SPPACS)  release  4.7.0  (tag  star_release_4_7_0)  was  created  under  this  contract 
to  serve  as  a  baseline  while  the  analysis  work  was  being  conducted.  This  allows  development  to 
continue  on  a  code  branch,  while  ensuring  that  stable  software  is  available  to  users.  The  branch 
usually  contains  newly  implemented  software,  which  may  not  be  stable  enough  to  allow  for 
operational  use. 


DRDC  CR  2005-126 


15 


5.  Track  Software  Issues 


Defect  Tracking  Systems  allow  users  to  keep  track  of  outstanding  bugs  in  their  product 
effectively.  STAR  and  SPPACS  issue  tracking  is  performed  using  a  web  accessible  tool  called 
Bugzilla.  This  tool  can  be  accessed  using  a  secure  web  interface  at  https://star.iotek.ns.ca.  Once 
the  appropriate  security  procedures,  detailed  on  the  web  page,  have  been  followed,  users  and 
developers  can  use  this  site  to  add,  view  or  modify  issues  related  to  the  software  packages.  If 
required,  the  Environmental  Acoustic  Data  Acquisition  (EADAQ)  and  Noise  Monitoring  Software 
packages  can  also  be  added  to  the  site. 

None  of  the  current  bugs  affect  this  call-up. 

A  breakdown  of  the  current  issues  for  the  STAR  and  SPPACS  distributions  are  shown  in  Table  3. 
The  total  number  of  unresolved  issues  is  shown  in  the  NEW/ASSIGNED/  REOPENED  column. 
The  total  number  of  opened  issues  is  broken  into  two  classes  of  severity.  Issues  classified  as 
BLOCKER/CRITICAL/MAJOR  are  issues  that  should  be  addressed  in  the  short  term.  Blockers 
are  always  addressed  immediately  to  ensure  that  the  user  community  can  continue  with  their  work. 
Issues  classified  as  NORMAL/  MINOR/TRIVIAL  are  issues  that  can  be  dealt  with  in  the  long 
term. 


Table  3.  Distribution  Issue  Summary  (29/03/2005) 


PRODUCT 

NEW/ASSIGNED/ 

BLOCKER/CRITICAL/ 

NORMAL/MINOR/ 

REOPENED  (TOTAL) 

MAJOR 

TRIVIAL 

SPPACS 

39 

1 

38 

STAR 

78 

4 

74 

The  following  gives  a  more  detailed  description  of  the  SPPACS  BLOCKER/  CRITICAL/MAJOR 
column: 

•  Issue  #  284  (major)  fails  DAT32  byteswap  case.  The  utility  is  attempting  to  read  the  extra 
gains  using  the  original  header,  which  may  be  in  a  different  byte-order  than  the  platform. 

The  following  list  gives  a  more  detailed  description  of  the  STAR  BLOCKER/ 
CRITICAL/MAJOR  column: 

•  Issue  #  193  (critical)  tacplot  does  not  cleanup  before  exiting.  The  work  around  for  this 
issue  is  to  ran  heap_gc  after  the  analysis  window  has  closed. 

•  Issue  #  107  (major)  problems  capturing  close  button.  A  solution  exists  for  this  but  has  only 
been  incoiporated  into  the  tactical  plot. 
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Issue  #  295  (major)  capture  screen  doesn’t  work  correctly.  The  Capture  Screen  button  does 
a  screen  capture  on  the  analysis  window  and  not  the  tactical  plot.  If  the  tactical  plot  is  closed 
and  reopened  this  will  work.  A  fix  is  available  in  the  next  release. 


•  Issue  #  329  (major)  overlays  need  to  be  optimized.  The  tactical  plot  is  too  slow  (on  bender). 
To  load  on  Bender  (dual  P4-  1  GHz),  it  takes  approximately  20  seconds  the  first  time  and 
15-20  seconds  thereafter. 
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Fastest  Fourier  Transform  in  the  West 
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Low  Frequency  Analysis  and  Ranging 
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RF 

Radio  Frequency 

RISO 

Regional  Individual  Standing  Offer 

SA 

Scientific  Authority 

SONLIB 

Sonar  Library 

SPLIB 

Signal  Processing  Library 

SPPACS 

Signal  Processing  Packages 
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Software  Tools  for  Analysis  and  Research 

USB 

Universal  Serial  Bus 

VLA 

Vertical  Line  Array 
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